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ABSTRACT 



The Cumberland Valley Girl Scout Council is funded primarily 
by the United Way of Middle Tennessee. Approximately 5 years ago, the United 
Way of Middle Tennessee adopted an "outcome -based investment model" whereby 
agencies receive funding based on their ability to specify and monitor 
program outcomes based on genuine client and community need. To adapt to the 
new outcomes-based funding environment, the Cumberland Valley Girl Scout 
Council developed a monitoring and evaluation process to identify outcomes 
for youth development. The monitoring/evaluation process was based on an 
integrated quantitative-qualitative design that was intended to provide 
contextual information for guiding program decision making. The process was 
revised continually in response to the following: time constraints, evolving 
capacity of the organization's staff to monitor their own programs, and new 
information being learned about girls in the programs being evaluated. The 
outcome statements identified during the monitoring/evaluation process 
diverged from the language of the United Way outcome statements but did not 
differ from much of the research literature on youth development. The 
council's experience suggests that strategies designed to improve 
organizational learning must be revised continually through a process of 
monitoring and evaluation so as to respond to what has been learned. 

(Contains 10 references.) (MN) 



***************************************************************************** 

* Reproductions supplied by EDRS are the best that can be made 

* from the original document. 



C£ o 7 3 f 3s - 



</-) 

m 

</-> 

o 



Q 



W 



Organizational Learning in the Nonprofit Sector: 

Lessons from Working with the Girl Scouts 

William R. Penuel 
Program Evaluation Coordinator 
Department of Research & Evaluation 
Metropolitan Nashville Public Schools 

March, 1997 

Paper presented at the Annual Meetings of the American Educational Research 

Association, Chicago IL 



Running Head: NONPROFIT ORGANIZATIONAL LEARNING 



send requests for additional copies to: 
Bill Penuel 
P.O. Box 40692 
Nashville TN 37204 
Phone: (615) 297-5923 
Email: bpenuel@mindspring.com 



U,S. DEPARTMENT OF EDUCATION 
OfUce of Educational Research and Improvement 
GCATIONAL RESOURCES INFORMATION 
CENTER (ERIC) 

This document has been reproduced as 
received from the person or organization 
originating it. 

Minor changes have been made to 
improve reproduction quality. 




• Points of view or opinions stated in this 
document do not necessarily represent 
official OERI position or policy. 




2 




"PERMISSION TO REPRODUCE THIS 
MATERIAL HAS BEEN GRANTED BY 




TO THE' (EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC).” 



NONPROFIT ORGANIZATIONAL LEARNING 



Abstract 

In recent years, management experts, educators and human development specialists 
have been concerned with creating “ learning organizations” (Senge, 1990) and “ learning 
societies,” (Keating, 1996), that is, human social institutions based on innovation and 
knowledge-building. This paper examines how one nonprofit organization attempts to 
become a learning organization as it adapts to a new funding process that requires greater 
attention to monitoring and to evaluation of program results. This organization's 
experience suggests that strategies designed to improve organizational learning must 
themselves be continually revised in order to respond to what is learned through the 
processes of monitoring and evaluation. Implications for how nonprofits might learn from 
such efforts and design effective monitoring and evaluation strategies are considered. 
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Learning Organizations and Learning Societies 

In recent years the concept of “organizational learning” has become widely 
accepted as the basis for structuring businesses and private organizations for the future 
(see e.g. Chawla & Renesch, 1995; Marquardt, 1996; Senge, 1990). While the idea that 
groups of people organized about the completion of a task, the delivery of a service, or 
the creation of a product can learn as a group is hardly novel, the ascendancy of 
organizational learning as a construct that guides managers and workers in the 1990s is, 
indeed, new. The ascendancy is not just a response to changes in management research 
and trends in business literature but to a host of complex transformations in the 
workplace: the “down-sizing” of businesses and elimination of middle management; the 
decline of Taylorism (mass production) as a model for the design of organizations; the 
emergence of total quality management as a principle of organizational development; 
and rapid technological change that has transformed how knowledge and information 
is used in organizations. 

At its root organizational learning is about the concept of creating and building 
knowledge within organizations (Hendry, 1996). According to Senge (1990), ’’the basic 
meaning of a ‘learning organization’ [describes] an organization that is continually 
expanding its capacity to create its future” (p. 14). It is generative, meaning that it 
“enhances our capacity to create” (Senge, 1990, p. 14). A learning organization does 
more than learn from experience; it continually reforms its processes and practices to 
realize more effectively the vision shared not just by managers but by all members of 
the organization. 

Human development researchers have also been interested in examining the 
creation of “learning societies,” to investigate how groups of people generate and build 
shared knowledge to advance key health and education projects. Like the growth of 
interest in organizational learning in business, the interest in developing social and 
educational institutions to promote a learning society emerges out of a particular set of 
socio-historical processes, including the rapid development of new technologies, the 
decline in fortunes of the poor and undereducated, and contemporary educational 
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reform practices (Keating, 1996). Schools and other public organizations dedicated to 
the promotion of human development have also had to respond to the change in the 
needs of the workforce, specifically, the need for a different kind of worker: 

The new economies may well depend on a much greater depth and 
breadth of population competence in order to function well. When 
limited skills were economically adequate, a high discard rate could be 
tolerated — economically, if not humanely. Failure to develop our 
human resources as fully as possible in the future, however, may hamper 
our potential to become a knowledge-based economy. (Keating, 1996, p. 
12 of ms) 

The inference Keating and others would have us draw is that education and 
other human development specialists must work to shape their organizations so that 
their own workers and the students they serve will become effective members of 
learning organizations. Those organizations that will survive in the twenty-first 
century economy, according to Keating (1996), will be those that learn how to 
promote innovation and build knowledge collaboratively in order to improve teaching 
and learning practice. They will be the organizations that recognize the “social nature” 
of knowledge and that develop structures for creating processes for making 
information available to shape and guide decision-making, thus creating a continuous 
movement from reflection to practice and back. 

To be sure, not all educators agree that this new kind of learning society 
comprised of learning organizations is one we should seek to develop. Advocates of 
so-called “basic skills” approaches argue that creative problem-solving and teamwork 
are skills that should be taught after direct instruction in reading, writing and 
mathematics has been delivered. Other educational researchers have warned that there 
are built-in contradictions between the idea of promoting the development of free, 
creative problem-solving individuals who are prepared to accept willingly work in the 
twenty-first century workplace as defined by corporate elites. 

Besides preparation for work there are many other goals of education and 
human development programs sought by social service workers, and the business-like 
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language of outcome-based investments masks these diverse goals. Moreover, creating 
a learning organization requires ongoing changes in the construction of the worker, the 
worker’s roles in the organization, and the place of monitoring, reflection, and 
evaluation in decision-making. This shift, it might be warned, is difficult and must be 
accompanied by shifts in the monitoring and evaluative processes of outcome-based 
investments if those processes are to be useful in decision making. 

These problems and challenges are ones I take up more focally in this paper, 
which explores how a nonprofit organization, the Cumberland Valley Girl Scout 
Council, takes on the challenge of becoming a “learning organization.” The context of 
this particular agency’s development is the organization’s adaptation to a new funding 
process called outcome-based investments (described below) begun by the local United 
Way, the primary funder of the agency. I write as an evaluator of programs of the Girl 
Scout Council, someone who was hired to assist the organization with meeting 
requirements of the new funding process, a process that places much greater weight on 
monitoring and evaluation of program results than did past funding processes. My 
comments, then, are those of an insider and a beneficiary of the outcome-based 
investments process. My analysis is intended to guide the further development of this 
process as a fair, productive means of supporting organizational learning in the 
nonprofit sector in my community. 

Outcome-based Investments As a Driver of Organizational Learning in Middle 
Tennessee 

One of the key drivers of organizational learning in the nonprofit sector 
nationally in the last five years has been the decline in funding. Smaller pools of 
community, state, and federal resources mean that agencies are in greater competition 
for funding for services. Large private funding agencies have been forced to devise 
efficient and fair ways to allocate these shrinking funds and to respond to community 
members’ desires to see more “results” from social services. One model that several 
communities have adopted is the “outcome-based investment model,” whereby 
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agencies receive funding based on their ability to specify and monitor outcomes of 
programs based on genuine client and community need. 

About five years ago the United Way of Middle Tennessee began investigating a 
shift toward just such a model as they allocated funds to community. While in the past 
agencies could often count on stable funding from United Way year after year, the 
funding agency was driven by economics and pressure from the community to 
investigate a way to allocate funds based more focally on outcomes, rather than on 
numbers of clients served, history of service in the community and organizational 
leadership. Over the past two years agencies have been asked to make the transition to 
having the vast majority of their United Way allocated funds be awarded through an 
outcome-based investment process. 

Just to survive, the shift in funding has required most service providers to 
become “learning organizations” practicing the five disciplines Senge (1990) describes. 
Direct service workers have had to re-tool their own jobs to include program 
monitoring. The models of service delivery used in the past— whereby success was 
measured by numbers of clients served and the sense that one has done a good job- 
have had to be revised. Staff have had to pull together to clarify their vision of how 
clients would function better with their agency’s services. And staff have had to 
rethink their relationship to their funding sources and to other agencies, pooling 
together to collaborate on projects they might have done on their own in the past. 

Each of the key elements and terms of outcome-based investments is intended 
to direct service providers’ proposals toward a particular view of those people who are 
being served, to the service provider, and to the funding agency. In fact, none of these 
latter terms (historically those associated with the nonprofit sector) is used to describe 
the key headings of a proposal. The developmental endpoints of clients or those being 
served are termed “outcomes;” those being served are called “customers”; services 
delivered are called “products,”; and the funders are called “investors.” On the way to 
outcomes “customers” are expected to reach “milestones” or intermediate outcomes 
designed to help them reach “performance targets.” 
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Central to the outcome-based investment program is monitoring and 
evaluation. In order to receive funding, an agency or collaboration must agree “to 
report milestones on a quarterly basis and participate in the evaluation process” 
(United Way of Middle Tennessee, 1996, p. iii). In addition, proposals are evaluated on 
the “strength and clarity of results,” which must be defined in terms of changes in the 
behavior of customers (United Way of Middle Tennessee, 1996, p. iii). Funding lasts 
for two years and is contingent upon ongoing monitoring of milestones and products. 
The proposal requires that agencies identify specific areas of behavior change among 
customers and specify what evidence will be used to verify that the behavior change 
has been achieved through the program. 

In order to ensure that the outcome-based investment process had buy-in from 
major providers in the area, United Way of Middle Tennessee created Councils for 
different service aras composed of service providers and community members. These 
Councils met to discuss and decide upon desired community outcomes in each service 
area, the kinds of services that would be funded, and some parameters for specific 
service areas. These groups met several times with United Way staff who provided the 
outline for proposals and training in the outcome-based investment application process 
to all interested agencies. 

A central problem for United Way, however, has been buy-in to the 
monitoring and evaluation of programs. Agencies have struggled to learn how to 
define their program results and customer milestones in quantifiable terms. They have 
had to revise data collection processes and invent new forms for gathering information 
on clients. They have seen proposals denied funding for inadequate evaluation designs. 
To many direct service workers, this part of the process remains a mystery. 

The field of service agencies that has emerged with funding from United Way is 
comprised of agencies that closely resemble the type of organization that Keating 
(1996) describes as “adaptable” to twenty-first century conditions— innovative and able 
to build knowledge about practice within the organization. Many agencies have seen 
their United Way funding fall, but others have been awarded more funds to implement 
programs with specific outcomes and monitoring processes established. While the 
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Cumberland Valley Girl Scout Council saw its own funding drop this year as a result 
of decisions of councils, from the inside it appears to be one of the more successful 
organizations in using evaluation to guide programming and so stands as a fairly 
representative adaptive learning organization. 

Along the way, key issues have been raised in girls programming at the 
Council. For example, through the monitoring and evaluation processes, 
developmental outcomes that diverge from the United Way’s agreed-upon ideal 
“outcomes” for youth development have been identified. Also through this process, an 
integrated quantitative-qualitative design, intended to provide rich, contextual 
information that could guide decision making for programs, had to be continually 
revised to adapt to time constraints, to the evolving capacity of staff to monitor their 
own programs, and to the new information being learned about girls in the programs 
being evaluated. 

Divergent Developmental Outcomes for Girls: Perspectives of Girl Scouts’ Staff 
and Volunteers 

The Cumberland Valley Girl Scout Council, the organization with which I am 
primarily concerned here, was a part of the Youth Development area of the 
Strengthening Families Council. This Council was responsible for defining a set of 
outcome statements that agency proposals would need to address in order to receive 
funding under the outcome-based investment process. The outcome statements that 
this group defined were: 

A community in which: 

• all youth will have services that enable them to successfully complete 
[sic] high school or its equivalent and to be self-sufficient 

• all youth are prepared and motivated to be contributors to their home, 
school, and community 

• all youth will have positive and diverse role models 

• all youth will believe in their own potential 
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• all youth will have the ability to make healthy life choices 

• social and recreational opportunities are available to all youth (United 
Way of Middle Tennessee, 1996, p. 35) 

These outcome statements suggest that agencies are to promote the development of 
youth as more free decision-makers who are adapted to mainstream institutions such as 
the school and involved in fun activities for their own sake. These goals are not 
different from much of the research literature on youth development (e.g., Dryfoos, 
1990) and are well within the mainstream of expectations of youth named as part of 
national education and health goals in such documents as Healthy People 2000, 
Prevention Plus III and Goals 2000. 

In the course of interviewing staff and volunteers of the Girl Scout Council 
about their own goals for girls, I found goals that diverged from the language of the 
United Way outcomes statements. Based on the premise that learning organizations 
share a common vision or goal (Senge, 1990), I wanted to know whether volunteers 
and staff described their activities with girls in terms of the current stated goals of the 
Girl Scout Council. These goals correspond closely with those of United Way’s 
Strengthening Families Council: promoting girls’ self esteem; helping girls learn values 
and ethics; teaching life skills, such as making healthy decisions; leadership; and service 
to the community. 

I learned, however, that staff and volunteers have at least two other sets of 
understandings of ideal developmental outcomes for girls. One set of volunteer leaders 
and staff take primarily an “Adjustment” view of what Girl Scouts is. This view 
provides a more traditional interpretation of the United Way goals of what it means to 
“contribute” in terms of upholding traditional values of church, patriotism, and family 
loyalty. These leaders are likely to involve their girls in service activities such as 
visiting a veteran’s cemetery or holding a flag ceremony. These activities underscore 
girls’ obligation to respect their national heritage and to show loyalty to their country. 

These same adults may also work to help girls to develop obedience to the 
authority of church and other community leaders. Their “success stories” for girls 
typically involve stories about girls who are initially rambunctious, disobedient, or 
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unruly and who gradually learn to listen to the adults in the troop. In this way, they 
take “adjustment” to the prevailing community and religious norms as the primary 
goal of Girl Scouting. 

By contrast, other Girl Scout staff and volunteers see as one of their primary 
goals the widening of opportunities for girls. They take what I term a “Social Change” 
view of Scouting. The point for them is not to help girls adjust to a society that limits 
their opportunities in work and social life but to provide to girls opportunities that 
widen the circle of what they imagine is possible. Their view of Scouting is not just 
about serving the community but about changing it so that all girls have a better 
chance at succeeding at school, work, and play. 

Staff and volunteers who adopt this view of Scouting may select activities for 
girls that are quite different from those adopting the “Adjustment” view. For example, 
during the first year I was evaluator, I observed troops who enacted a play written 
about the women’s suffrage movement that culminated in the Tennessee Legislature’s 
being the thirty-sixth state to ratify the amendment that resulted in giving women the 
right to vote. In addition, the Council sponsored a conference on education for girls 
and invited David Sadker, an educational researcher who has documented the ways 
that girls have been inequitably treated in our nation’s public schools (Sadker & 
Sadker, 1994). 

It is important to note that while these divisions were understood by some at 
the Council to have been present for some time, it is the monitoring and evaluation of 
outcomes that yield empirical support to the existence of these divergent 
developmental goals held by staff and volunteers at the Council. Moreover, these 
motivations appear to be central to how staff and volunteers understand their 
involvement in the Girl Scouts. In other words, it is more important for the Girl 
Scouts as a “learning organization” to understand how staff and volunteers orient 
themselves with respect to this developmental difference (Adjustment View versus 
Social Change View) than simply to whether or not they understand the Council’s five 
goals or the United Way’s outcome statements. 
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Neither the Council’s nor the United Way’s “official” documents capture what 
motivates adult participants in Scouts. For the Girl Scouts to continue to develop as a 
learning organization, it is critical for staff and volunteers to examine more closely the 
actual models that they use to guide their programming decisions for girls. More 
generally, organizational learning in the outcome-based investment process may 
require “thinking outside the box” of the stated program outcomes to ask the 
questions: What do those implementing our programs really believe about the services 
they are providing? What motivational differences exist across the organization for 
delivering services? And finally, do those differences need to be reconciled, or can they 
exist in harmony as the organization continues to learn how to improve its service to 
clients in a way that is based on agreed-upon elements of a shared vision? 

Adapting Monitoring and Evaluation to Meet Evolving Program Needs 

One additional development within the monitoring and evaluation processes of 
outcome-based investment is the need to revise continually the research process to 
meet the emerging needs of programs for decision-making. Over the course of the past 
two years, as an evaluator I moved from a labor-intensive quantitative-qualitative 
design of troop and adventure programming to a quantitative design with frequent 
monitoring of results and mini-studies to examine more closely relationships in the 
quantitative data. This latter design, I argue here, is one that is the most feasible for 
other nonprofits that seek to integrate data and information into program decision 
making at a low cost to the organization. It is, however, not the “final” answer to the 
problems of monitoring and evaluating programs; rather, monitoring and evaluation 
processes themselves must undergo continuous improvement and learning as an 
integral part of nonprofits becoming learning organizations. 

My evaluation work began two years ago and initially focused on two different 
types of Girl Scout programming. The first evaluation project focused on the troop 
experience for girls. My task was to describe the developmental processes of troop 
activities and to evaluate their effectiveness with respect to the five goals of the Council 
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(improving self-esteem, teaching ethics/ values, teaching life skills, promoting 
leadership, and providing opportunities for service). In the second project I was asked 
to evaluate the effectiveness of a school-based program called Girls High Adventure 
(GHA). GHA was designed for “at-risk” girls in middle school and included a self- 
exploration curriculum that culminated in a weekend with adults that involved 
outdoor challenges and opportunities such as a ropes course, horseback riding and 
camping. 

My design for both projects included a mix of quantitatively-based surveys and 
qualitative observations and interviews designed to provide rich, contextual 
information about program success (or failure) and what might be contributing to it. 
My assumption, with that of Keating (1996), is that developmental data for decision- 
making must be sufficiently rich and detailed so it can speak to speak not only to the 
what of development but more importantly to the how of development. My troop 
evaluation, therefore, involved numerous observations at four distinct troop 
environments— a rural area, a wealthy suburb, a middle class suburb, and an inner city 
area. The observations and discussions with girls and troop leaders there were 
intended to provide an account of the diversity of activities, goals, and effects of troop 
participation for the adult and girl participants. Similarly, the evaluation of GHA 
initially involved periodic observations of the classroom experiences of girls, review of 
journal entries of girls in the program and interviews with participants about their 
experience. Both program evaluations used quantitative data primarily to supplement 
the information being generated by the qualitative report. 

I provided the Executive Director and the Assistant Director with monthly 
summaries of findings and short papers detailing what I was learning in the troop 
settings and in GHA. From time to time these formed the basis of discussion for staff 
about proposed changes in the programs, and preliminary quantitative data were used 
by the staff grant writer to prepare the upcoming year’s outcome-based investment 
application. A final report on troop effectiveness and on GHA was produced in the 
summer after the first year. 
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In the long run, this first year’s design proved to be less than feasible. First, as a 
part time consultant, I found it nearly impossible to attend enough meetings of troops 
for a sufficiently wide variety of types of troops to develop a deep understanding of the 
learning processes of troops. Second, I believe that the design I used was not well 
understood by all the troops themselves, and it was not clear to me that they benefited 
from the process of being the participants in the evaluative process. Third, the Council 
needed more quantitative data to support its claims as to troop effectiveness for funders 
such as United Way. There were too few girls at the same age level with similar 
demographic backgrounds in the to make any claims about the effectiveness of Scouts. 

During the second year I have revised this process considerably, both to meet 
the needs of the Council for assessing troop effectiveness and to provide timely, 
contextual information for a more limited array of questions. Currently, I use the 
much larger database of students from the GHA program to assess the difference on a 
range of program outcomes between girls in GHA who have previously been a part of 
a Girl Scout troop and those who have not. These data have pointed out that on the 
whole, girls in the program who have had Scouts have slightly higher self-efficacy and 
have significantly higher connections to prosocial adults and activities (such as 
community service) than do their peers who have never been in Scouts. 

In addition, through an analysis of survey data on expectations of the future, I 
found that girls from a private school had much more developed notions of their 
future career paths than did a comparable sample of public school students. This 
finding has led to discussions of how to integrate career information into the GHA 
curriculum and to a more in-depth study of girls’ conceptions of their future and how 
they intend to reach their long term goals. 

For a number of reasons, this year’s design has proven more feasible and more 
productive from the researcher’s point of view. First, it is more cost-effective than the 
extensive qualitative troop design. It requires much less time spent traveling from 
troop to troop and in transforming qualitative observations into coherent narratives of 
development. Second, the design meshes well with staff’s increased capacity to 
monitor and evaluate their own results more effectively. More staff currently 
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understand the outcome-based investment process, and in the GHA program, training 
in data collection by the program manager is integrated each semester into volunteer 
training for mentors. Finally, the design allows for flexible adaptation of the 
evaluation process to address gaps in data and gather more contextual information on 
aspects of girls’ development as those needs arise. 

Concluding Comments: The Future of Outcome-based Investments 

Outcome-based investments remains a controversial process in the Nashville 
area despite its community-wide implementation by a major funder. Many agencies 
have seen their funding drop significantly, and they are disappointed in the cutbacks in 
services they have had to make. A few agencies, like the Girl Scouts, have adapted 
fairly well to the changes, but still struggle to find new ways to thrive in the funding 
climate of the late 1990s. What, then, will enable the outcome-based investment 
process to be a positive catalyst for creating learning organizations in the nonprofit 
sector? 

I believe that the lessons from the Girl Scouts are instructive here. First, it is 
critical that organizations be able to make sense of “outcomes” along a diversity of 
developmental pathways. For some, being a service worker means living out one’s 
most deeply-held religious values, while for others, it is a means of promoting 
community change. For only a few is service an opportunity to engineer 
mechanistically the new twenty-first century worker. For service workers to buy in to 
outcome-based investments, the process must acknowledge these motivations for 
service and to make room for a variety of developmental goals. The inclusiveness of 
the process of defining outcomes ensures that the United Way’s process is likely to 
continue to evolve toward this goal, but facilitators will need to be sensitive to the 
variety of goals in the community, even when those stakeholders are not represented 
in the decision-making representatives. 

Second, the outcome-based investment process will need to take much greater 
care to devise ways to assist agencies and organization with monitoring and evaluating 
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programs. Few organizations currently have the capacity internally to conduct their 
own program evaluations; those that do must do so at as low a cost as possible, so 
funds are not diverted from programming. An outcome-based investment process and 
the re-structuring of the nonprofit workplace it implies requires that direct service 
workers themselves be involved in monitoring and evaluation. Specific supports and 
resources must be devoted to helping such workers define their own roles and interests 
in the process. It is essential to provide training in how to use the data and 
information gathered so programs can have ongoing feedback about their program’s 
success. 

The outcome-based investment process is neither a panacea nor an isolated 
strategy. As a driver of organizational learning, it has the potential to transform how a 
community provides services to its citizens, but only insofar as the process itself is a 
learning process — one that adapts to the changing needs of the community and those in 
it. In short, “organizational learning” is not itself a fixed concept, but one that must 
evolve as processes designed to promote creativity, knowledge-building and innovation 
in organization change the very contexts of organizational practice. 
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